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METHOD AND SYSTEM FOR IDENTIFYING ERRORS IN COMPUTER SOFTWARE 

Background Of The Invention 
Field of the Invention 

[000 1 ] This invention relates to analyzing errors in computer programs. More 

specifically, the invention relates to identifying errors in computer programs and to specifying 
test cases that will cause the errors to occur. 

Background Art 

[0002] There are several known methods for identifying errors in software programs. 

The most common of these methods is testing; that is, executing the target program under 
different conditions in order to confirm its correct behavior. Another group of methods, 
program verifiers, attempt to prove mathematically that a program's behavior is correct for all 
possible input conditions. In addition, there are static analysis tools that perform a set of 
limited checks without executing the program. These tools report some possible errors. 

[0003] There are numerous testing tools for exposing errors in a software program. 

With these tools, the user is provided with an environment for specifying program values, for 
executing the program in steps, and for examining the resulting program values. 

[0004] A program verifier attempts to prove that a formal assertion of correct 

behavior is true for all possible inputs and for all possible execution paths. If the proof is 
successful, there are no errors represented by the assertion in this portion of the program. If 
the assertion is proved false, a set of conditions is identified that specify the program's 
erroneous behavior. Program verifiers are discussed, for instance, in "Predicate Abstraction 
For Software Verification," by Cormac Flanagan and Shaz Qadeer, Proceedings of the 29 th 
ACM SIGPLA-SIGACT symposium on Principles of programming languages, Vol. 37, Issue 
1, January 2002. 
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[0005] Another class of tools, referred to as Weak Status Analysis tools, use syntactic 

and data-flow analysis techniques found in compilers to identify possible program errors. 
These techniques do identify some program errors, but do not consider the Boolean 
conditions controlling a program execution and therefore also produce some false error 
reports. 

[0006] Other static analysis tools, referred to as Strong Static Analysis tools, use 

additional theorem proving and symbolic execution techniques to reduce the number of false 
error reports and to specify the program conditions that lead to an erroneous program 
execution. One known Strong Static Analysis tool is the Beam analysis tool The Beam 
analysis tool is discussed in "A Software Falsifier," by Daniel Brand, International 
symposium on Software Reliability Engineering, pp. 174-185, October 2000. 

[0007] While there is a wide range of existing tools to identify program errors, each 

approach has its limitations. 

[0008] More specifically, while testing is the most common method for finding errors, 

it is known that the effectiveness of testing is limited by a user's ability to create new test 
cases that explore a program's different behaviors. It is normally not feasible to test all 
possible program executions and is extremely difficult to know what area of a program's 
execution to explore. 

[0009] Verifiers require precise and detailed specifications of the target program's 

correct behavior as well as the surrounding environment of the program execution. Still, it is 
often the case that an assertion about a program's correctness cannot be shown to be either 
true or false. This may be due to the limitations of the verifier or of the specification. User 
assistance is usually needed to guide and to attempt to complete the proof. 

[0010] Weak Static Analysis tools usually require no additional user input or 

specifications, but typically produce a large number of false error reports. This is because 
these tools do not analyze the Boolean conditions that determine a program's control flow, 
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and therefore usually are uncertain if the error will actually occur in the program's actual 
execution. The user must sort through the false errors to find the real errors reported. 

[0011] The additional theorem proving and symbolic execution techniques used in 

Strong Static Analysis tools, such as the Beam tool, do reduce the number of false error 
reports. However, no method can resolve all conditions in a program and false error reports 
are still a possibility. For instance, when the Beam analysis tool cannot determine if a 
particular error will occur in actual program execution, this tool chooses not to report the 
potential error. This conservative policy has the effect of greatly reducing the number of false 
error reports, but at the expense of ignoring some potential errors. 

Summary Of The Invention 
[0012] An object of this invention is an improved method and system for identifying 

errors in computer programs. 

[00 1 3] Another object of the invention is to use the strengths of existing methods to 

produce a new and effective method for exposing errors in computer programs that would not 
likely be found using any one existing method. 

[0014] These and other objectives are attained with a method and system for 

analyzing a computer program. The method comprises the steps of analyzing a computer 
program to generate an initial error report and a list of suspected error conditions, and 
generating a set of assertions and inserting the assertions into the computer program to 
determine if the suspected error conditions are valid. 

[00 1 5] With the preferred embodiment of the invention, described in detail below, a 

strong static analysis method that analyzes the Boolean conditions determining a program's 
execution, such as the Beam analysis tool, is used to identify an initial set of error reports for 
a target program. When the strong static analysis fails to determine if the condition is true or 
false, the condition along with the potential program error is captured to form a suspected 
error. 
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[00 1 6] With prior art systems, normally these suspected errors would not be reported, 

to avoid false error reports. In contrast, in the preferred embodiment of this invention, these 
suspected errors are directed to an assertion generator to produce a monitor - that is, source 
code modification that is integrated with the original program. This and other inserted 
monitors check the conditions for the suspected error during the program execution and report 
the error if the conditions are valid. 

Brief Description Of The Drawings 
[0017] The foregoing and other objects, aspects, and advantages of the invention will 

be better understood from the following non-limiting detailed description of preferred 
embodiments of the invention, given with reference to the drawings that include the 
following: 

[00 1 8] Figure 1 is a block diagram describing the current environment for testing 

programs with test cases to produce an error report. 

[00 1 9] Figure 2 is a block diagram describing the current weak static analysis process, 

which yields an error report. 

[0020] Figure 3 is a block diagram describing the current strong static analysis 

process, which yields an error report and a set of potential error conditions. 

[0021] Figure 4 is a block diagram of the preferred embodiment of the invention. 

[0022] Figure 5 is a block diagram describing the current verification process, which 

yields an error report and a set of suspected errors. 

[0023] Figure 6 is an example input produced by the invention. 

[0024] Figure 7 shows a modified program that includes monitors for the three 

potential errors in the program of Figure 6. 
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Detailed Description Of The Preferred Embodiments 
[0025] This invention, generally, provides an improved method and system for 

identifying errors in programs by using the strengths of existing methods. 

[0026] Figure 1 illustrates a typical program testing process. A program 12 is entered 

into a program execution environment 14 along with a set of test cases 16. Test cases 16 can 
be provided by an external file or interactively by the person doing the testing. Each test case 
provides values for selected program variables and an expected behavior for the program. 
The program is executed and an error report 18 is generated confirming correct program 
behavior or indicating that an error has occurred. 

[0027] Figure 2 is a block diagram describing the current weak static analysis process. 

A program source code 22 is input to the weak static analysis process 24 and an error report 
26 is generated that indicates a set of potential errors. 

[0028] Figure 3 shows a program source code 32 as input to the strong static analysis 

process 34 and the resulting error report 36. In addition, the strong static analysis process has 
been modified to produce a list of suspected errors 38 that have uncertain feasibility, due to 
the limitations of the analysis of the Boolean conditions that determine if the error would 
actually occur in normal program execution. The suspected error list is a list of errors and the 
conditions that must be true for the error to occur. 

[0029] Figure 4 describes the preferred embodiment of the invention. A first program 

42 is analyzed with strong static analysis 44 to identify one set of errors in error report 46. 
The strong static analysis also produces a set of suspected errors 48 that are input to an 
assertion generator 50 that, in turn, produces a set of source code statements or monitors and 
inserts these monitors into program 42 to form second program 52. As represented at 54, 
program 52 is then executed normally or with user supplied test cases 56. During the 
execution of program 52, the conditions for the suspected errors are monitored. If the 
conditions for one of the suspected errors are satisfied, then the added assertion statements 
generate a second error report 58 indicating that the error has occurred. 
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[0030] The Assertion Generator 50 can be extended to insert a call to an error 

handling routine after reporting the error, but before the actual error occurs. This extension 
could be used in testing for multiple errors in one run. 

[003 1] Optionally, the Assertion Generator 50 could be given the complete list of 

errors, both the reported errors of Error Report 46 as well as the suspected errors. It would 
then produce monitoring statements to track conditions for all these errors and report them if 
they occurred during testing or normal execution. This would provide additional 
confirmation of the strong static analysis and if error handling is provided, could prevent 
known errors from interfering with the testing for additional errors. 

[0032] Figure 5 is a block diagram describing a typical verification process that has 

been modified to generate a list of suspected errors. A program 62 is input into the program 
verifier 64 along with a specification 66 of the program's correct behavior and a description 
68 of the program's execution environment. In addition to producing an error report 70 that 
confirms correct program behavior or that lists program errors, this verifier 64 also produces a 
list of suspected errors 72 that it could not confirm or deny through its analysis. Such a 
modified verifier can be used instead of the strong static analysis process included in the 
preferred embodiment shown in Figure 4. 

[0033] Figure 6 illustrates an example program 42 that may be input to the process 

described in Figure 4. A strong analysis program such as the Beam analysis tool would detect 
three potential errors: 

1 . At label LI , X will be used but un-initialized if both P(A) calls return false. 

2. At label L2, division by 0 occurs if the second call to P(A) returns true. 

3. At label LI, Division by 0 occurs if the first call to P(A) returns true and the 
second call returns false. 

[0034] The Beam analysis tool would report the first two errors, but not the third. 

This analysis tool would assume that it is unlikely that a function P would return different 
values when called with the same argument and not report a problem, to avoid a false error 
report. Thus, the third potential error is an example of a suspected error. 
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[003 5] The Assertion Generator 50 of Figure 4 would accept the list of suspected 

errors and produce a modified program 52 shown in Figure 7. In this example, all errors 
including the errors reported in Error Report 76 as well as the suspected errors are passed to 
the Assertion Generator 50. Figure 7 shows monitors of the conditions for the three errors 
and three print statements for reporting each error if it occurs during testing or normal 
program execution of Program 52. The Assertion Generator 50 can be extended to also call 
an error handling routine to avoid executing the erroneous statement when it is detected. 

[0036] While it is apparent that the invention herein disclosed is well calculated to 

fulfill the objects stated above, it will be appreciated that numerous modifications and 
embodiments may be devised by those skilled in the art, and it is intended that the appended 
claims cover all such modifications and embodiments as fall within the true spirit and scope 
of the present invention. 
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